Event detection with concurrent data updates

ABSTRACT

An event detection system allows data to be inserted while event conditions are being checked. Each record is assigned a time stamp as it is inserted into a database. Each event condition check is assigned a time stamp range. The event condition check then produces only those matches that have at least one record with a time stamp in the range and no record with a time stamp after the range. After each event condition check, the range is changed so that, in subsequent checks, no part of a previous range is duplicated and no time stamps are excluded from every checked range. As a result of this process, records may be inserted while event conditions are being checked.

FIELD OF THE INVENTION

[0001] The present invention relates generally to computer processes,and, in particular, event detection by a programmed computer.

BACKGROUND OF THE INVENTION

[0002] A computer-assisted event detection system receives data, checksthe data to see if it To satisfies pre-selected “event conditions,” andthen outputs the data that satisfies the event conditions.

[0003] For example, an event detection system may connect buyers andsellers in an electronic marketplace. The system could receive two typesof data:

[0004] 1. buyer records that each contain the following fields:

[0005] a. description of a desired item

[0006] b . a desired price at which to buy the item

[0007] c. contact information about the prospective buyer

[0008] 2. seller records that each contain the following fields:

[0009] a. a description of an item for sale

[0010] b. a desired price at which to sell the item

[0011] c. contact information about the prospective seller

[0012] The records are preferably stored in a database in the memory ofa programmed computer, with the buyer records in one table and theseller records in another table. An example of an event condition couldbe that a buyer record and a seller record describe the same item,perhaps further qualified by the condition that the buyer's price be atleast as high as the seller price.

[0013] In general, the set of records that satisfies an event conditionis called a “match.” In this example, a match is a set that includes abuyer record and a seller record. The match represents the “event” inwhich the condition was satisfied, namely, that a buyer and a sellerhave agreed on a price for an item, as indicated by the buyer's andseller's records. Thus, the purpose of an event detection system is toascertain whether there are any events, that is, matches, that are notempty sets (sets with no records) but are sets that contain data recordsthat satisfy the pre-selected condition.

[0014] Event detection systems have wide application in commerce,especially on the internet in so-called e-commerce, in governmentadministration such as, for example, checking criminal records andfingerprints, and in transportation of goods and passengers.

[0015] In an event detection system, no match should be overlooked, but,rather, each match should eventually be found and output, but preferablyonly once. One way to ensure that a match is not output multiple timesis to design the event detection system so that it (1) waits for arecord, (2) checks for all possible matches that include the new record,(3) outputs any such matches, and then (4) resumes waiting for the nextrecord.

[0016] However, this design approach has several disadvantages. First,records that arrive during event condition checks cannot be insertedinto the database until the checks are complete. In other words, recordinsertion and event condition checks cannot take place “concurrently.”

[0017] Second, it is not possible to insert multiple records betweenevent condition checks. Thus, it is not possible to delay eventcondition checks when there is heavy demand for record insertion toallow for insertion of all records received between condition checks.

[0018] Third, this design forces all event conditions to be checked withthe same frequency.

[0019] By extending each record by one field per event condition,however, multiple records can be inserted between event condition checksand checked for event conditions at different frequencies. As eachrecord is inserted, it can be marked as “unchecked” for each eventcondition. To conduct an event condition check, then, the systemidentifies all matches with at least one record marked “unchecked.”These matches are then output and all records previously marked“unchecked” during the event condition check can then be changed to“checked.”

[0020] This approach also has drawbacks. Records that arrive duringevent condition checks still cannot be inserted until the checks arecomplete. In other words, this design approach still does not permitconcurrent record insertion. Also, adding new event conditions may bedifficult because it would require adding a corresponding marker fieldto many existing records. Although this problem may be avoided by usinga single marker field for all event conditions, the design of this eventdetection system still forces all event conditions to be checked withthe same frequency.

[0021] In designs that permit the insertion of multiple records intodatabases between event checks, a method is needed to identify thematches with at least one “unchecked” record, i.e., matches that havenot been previously output. An obvious and straight-forward approach isto produce all matches, then check each match, and output only thematches with at least one “unchecked” record. Unfortunately, this methodbecomes increasingly inefficient as the number of previously outputmatches increases. Another approach is to produce all matches that have“unchecked” records in each table in the database. This methodinevitably produces duplicates of matches that have “unchecked” recordsin different tables. Extra computation is therefore required to identifythese duplicates in order to avoid having multiple copies of the samematches as output. Thus, there remains a need for an event detectionsystem in which (1) records can be inserted while event conditions arechecked, (2) event conditions can be checked at different frequencies,(3) event conditions can be added and deleted without changing thestructure of the database, and (4) event condition checks do not includeproducing previously output matches or producing duplicate matches.

BRIEF SUMMARY OF THE INVENTION

[0022] According to its major aspects and briefly recited, the presentinvention is an event detection system that allows multiple data recordsto be inserted into a database and periodically checked to see if eventconditions are satisfied while also allowing the checking of eventconditions concurrently and at different frequencies. The present systemallows event conditions to be added and deleted without changing thestructure of the database. The system uses a method of event conditionchecking that produces each event condition match exactly once, withoutproducing duplicate matches or discarding matches.

[0023] The invention operates as follows. Each data record is augmentedwith a “time stamp.”Each event condition check has a corresponding rangeof time stamps. (For each event condition, each successive check has asuccessive range of time stamps.) A record with a time stamp below thetime stamp range is an “old record” with respect to the event conditioncheck. Likewise, a record with a time stamp within the range is a“current record,” and a record with a time stamp above the range is a“new record.” Each event condition check produces only matches with atleast one current record (to avoid reproducing matches from previouschecks) and with no new records (to avoid producing matches that will beproduced by later checks.)

[0024] Processing event checks while new records are being addedconcurrently can be done because every record inserted while an eventcondition is being checked will have a time stamp that identifies it asa “new record” and therefore not to be included in matches produced bythe check of current records. Use of a time stamp avoids ambiguity aboutwhich records were checked and thus which matches should be produced bywhich event condition checks.

[0025] Processing multiple event condition checks concurrently ispossible because each event condition check has its own range associatedwith it. The range defines which records are old, current and new.

[0026] The use of time stamps is thus a major feature of the presentinvention. However, other features and their advantages will be apparentto those skilled in the art of event detection from a careful reading ofthe Detailed Description of Preferred Embodiments, accompanied by thefollowing drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

[0027] In the figures,

[0028]FIG. 1 is a schematic diagram of an event detection systemaccording to a preferred embodiment of the present invention; and

[0029]FIG. 2 is a schematic diagram of an alternative event detectionsystem according to an alternative embodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0030] The descriptions below relate to the embodiments illustrated inFIGS. 1 and 2. The diagrams display subsystems and data flows. Thesystem diagramed in FIG. 1 is a simpler embodiment than the alternativesystem diagramed in FIG. 2.

[0031] The term “record” refers to data that is structured; the data maybe organized in one or more fields that are expected to contain specifictypes of information. Records will be intended for insertion into one ormore tables of data in a database. If two or more records contain datathat meet the pre-selected criteria, they will “match.” The existence ofa found match is an “event.”

[0032] The following description is organized by subsystem. If asubsystem has multiple processes, the processes can run concurrently. Onthe other hand, some subsystems maintain variables that may be accessedby multiple processes. Multiple processes cannot access the samevariable concurrently. If one process is executing a step that involvesaccess to a variable, and a second process is set to begin executing astep that involves access to the same variable, then the second processyields until the step of the first process completes execution.(Likewise, third and further processes will also yield while waiting forthe process with access to complete a step.) In the same manner, no morethan one process at a time has access to any single beginning or end ofa data flow. If a value arrives at the end of a data flow and no processis waiting to receive it, then the value is maintained until a processreceives it. If other values arrive during the wait, the subsequentvalues form a queue. When a process receives a value, the value isremoved from the queue.

[0033] Referring now to FIG. 1, a time stamp manager provides a timestamp for each record received. Note that a time stamp need not berelated to actual time in any way, but merely to a sequence of valuesthat increment (increase or decrease) with time. It is also not requiredthat the values change as a function of time. Any values that can becompared to each other to determine which are higher (later in time) andlower (earlier in time) may be used as time stamps as long as the valuesincrement so that no two records will receive the same time stamp. Forthe present example, it will be assumed that the time stamp valueincreases with passing time.

[0034] The time stamp manager has a single process, which is simply torepeat the following three steps:

[0035] 1. Wait to receive a record.

[0036] 2. Augment the record with a time stamp that is greater than anyprevious time stamp applied to a previous record.

[0037] 3. Send the record as augmented with its time stamp to a databasemanager.

[0038] An event condition manager 20 has two processes. One of these twoprocesses manages information about the time stamps correlated to therecords inserted in the database. The other process manages thescheduling of event condition checks.

[0039] The first process, an information management process, maintains atime stamp variable named “latest.” Initially, the “latest” variable hasa time stamp that is less than any time stamp issued by time stampmanager 10. This time stamp is called the “zero” time stamp. Theinformation manager process repeats the following steps:

[0040] 1. Wait to receive a time stamp from the database manager.

[0041] 2. Replace the value of “latest” with the time stamp receivedfrom the database manager.

[0042] The other process, a scheduling management process, maintains aset of event conditions. For each event condition, the process maintainstwo time stamp variables: “old” and “new.”Initially, each “new” variablehas the “zero” time stamp. For each event condition, the process repeatsthe following steps:

[0043] 1. Replace the value of the “old” variable with the value of the“new” variable.

[0044] 2. Replace the value of the “new” variable with the value of the“latest” variable.

[0045] 3. Send the event condition and the corresponding “old” and “new”values to the database manager.

[0046] A database manager 30 has two processes. One process managesinsertion of records into a database. The other process manages eventcondition checks on the database. The process that manages recordinsertion repeats the following steps:

[0047] 1. Wait to receive a record, augmented with a time stamp, fromthe time stamp manager.

[0048] 2. Insert the record, with the time stamp, into the database.

[0049] 3. Forward the time stamp to the event condition manager.

[0050] The process that manages event condition checks repeats thefollowing steps:

[0051] 1. Wait to receive an event condition, with “old” and “new” timestamps, from the event condition manager.

[0052] 2. Find all matches among records in the database for the eventcondition that have at least one record with a timestamp greater than“old” and have no records with time stamps greater than “new.”

[0053] 3. Output each match found in the previous step.

[0054] Referring now to FIG. 2, the second, alternative embodiment of anevent detection system according to a preferred embodiment of thepresent invention is more complex than the one illustrated in FIG. 1.This system offers the advantage of more opportunities for concurrentlyhandling records and event conditions.

[0055] Records bound for different tables need not be inserted into thedatabase in the order in which they are received. Furthermore, multiplerecords may be inserted simultaneously, and multiple event conditionsmay be checked simultaneously.

[0056] Time stamp manager 40 has a timestamp variable named “current”with an initial value set at a “zero” timestamp. Time stamp managerperforms a process to supply time stamps to a database manager 50. Thereis another process to supply time stamps to an event condition manager60.

[0057] The process that supplies time stamps to database manager 50repeats the following steps:

[0058] 1. Wait to receive a request from database manager 50.

[0059] 2. Increase the value of the time stamp variable “current” andsend the increased value to database manager 50.

[0060] The process that supplies time stamps to event condition manager60 repeats the following steps:

[0061] 1. Wait to receive a request from event condition manager 60 fora time stamp.

[0062] 2. Increase the value of the timestamp variable “current” andsend the increased value to event condition manager 60.

[0063] For each event condition, event condition manager 60 has acorresponding process. Each process maintains two time stamp variables,named “old” and “new.” Initially, each “new” variable has the “zero”time stamp value. Each process repeats the following steps:

[0064] 1. Replace the value of the “old” variable with the value of the“new” variable.

[0065] 2. Send a request to time stamp manager 40 for a time stamp.

[0066] 3. Wait to receive a time stamp from time stamp manager 40.

[0067] 4. Replace the value of the “new” variable with the time stampreceived from time stamp manager 40.

[0068] 5. Send an event condition and its corresponding “old” and “new”values to database manager 50.

[0069] Delays may be introduced between iterations of the sequence ofsteps in order to adjust the frequency of event condition checks. Varieddelays for different processes may be used to vary the check frequenciesamong different event conditions.

[0070] For each table in the database, database manager 50 has acorresponding input data flow and two corresponding variables—a yes/novariable named “inserting” and a time stamp variable named “last.”Initially, each “inserting” variable has value “no,” and each “last”variable initially has the “zero” time stamp value. For each table,there is a corresponding process that inserts records into that table.In addition, there is a process that receives event conditions andlaunches processes to check event conditions.

[0071] Each process that inserts records into a table repeats thefollowing steps:

[0072] 1. Wait to receive a record from the input data flowcorresponding to the table.

[0073] 2. Set the value of the table variable “inserting” to “yes” whena record is received.

[0074] 3. Send a request to time stamp manager 40 for a time stamp.

[0075] 4. Wait to receive a time stamp from time stamp manager 40.

[0076] 5. Insert the record into the table in the database, augmentedwith the received time stamp.

[0077] 6. Replace the value of the table variable “last” with the valueof the received time stamp.

[0078] 7. Replace the value of the table variable “inserting” with “no.”

[0079] The process that receives event conditions repeats the followingsteps:

[0080] 1. Wait to receive an event condition, with “old” and “new” timestamps, from the event condition manager.

[0081] 2. Launch an event condition check process with the receivedevent condition and “old” and “new” time stamps.

[0082] Each event condition check process includes the following steps:

[0083] 1. Access the tables in the database related to the eventcondition to find all matches that have at least one record with a timestamp greater than “old” and have no records with time stamps greaterthan “new.” Before each access to a table, wait until the table variable“inserting” has value “no,” the table variable “last” has a time stampgreater than “new,” or both.

[0084] 2. Output each match found in the previous step.

[0085] The method described may be used to find all matches for an eventcondition that have at least one record with a time stamp greater than“old” and have no records with time stamps greater than “new.” In thepresent specification, these matches will be referred to as “currentmatches.” Each record with a time stamp no greater than “old” will bereferred to as an “old record.” Each record with a time stamp greaterthan “old” and no greater than “new” will be referred to as a “currentrecord.” Finally, each record with a time stamp greater than “new” willbe referred to as a “new record.” Current matches have at least one“current record” and no new records.

[0086] Current matches may be collected by the following method. First,the tables from which records are to be combined into matches define a“list of tables.” For each table in the list of tables, the systemcollects each match that has (1) a current record from the table, (2) anold record from each previous table in the list, and (3) an old orcurrent record from each subsequent table in the list. The combinedresults are the “current matches.”

[0087] In the present specification, matches in which all records havetime stamps no greater than “old” are referred to as “old matches.”Matches in which a record has a time stamp greater than “new” arereferred to as “new matches.” The method is efficient in the sense thatit does not produce old or new matches, only current matches, and itdoes not produce duplicates of current matches.

[0088] The following process is one way to implement the method. Let “n”be the number of tables from which records are to be combined inmatches. Refer to the tables as Table 1, Table 2, . . . Table n.Initialize a variable “i” to value 1 and repeat the following steps “n”times:

[0089] 1. Add to the set of matches those matches that have:

[0090] a. an old record from each table in the list: Table 1, . . . ,Table i−1,

[0091] b. a current record from Table i, and

[0092] c. an old or current record from each table in the list: Tablei+1, . . . , Table n.

[0093] 2. Increase i by one.

[0094] If a list of tables as written in the method includes a Table 0or a Table n+1, then the list is empty.

[0095] For example, to collect current matches for an event conditioninvolving records from four tables:

[0096] 1. Collect matches with a current record from Table 1 and an oldor current record from each of Table 2, Table 3, and Table 4.

[0097] 2. Add to the matches those matches that have an old record fromTable 1, a current record from Table 2, and an old or current recordfrom each of Table 3 and Table 4.

[0098] 3. Add to the matches those matches that have an old record fromeach of Table 1 and Table 2, a current record from Table 3, and an oldor current record from Table 4.

[0099] 4. Add to the matches those matches that have an old record fromeach of Table 1, Table 2, and Table 3, and a current record from Table4.

[0100] Those familiar with the art will realize that these systemproperties may be realized in a variety of implementations in additionto those described here. Note that time stamps, as used here, need notrelate to time. Time stamps may be drawn from any set of objects orquantities for which subsets may be expressed. For example, time stampsmay be numbers. Also, a time stamp range may consist of any subset ofthe set from which time stamps are drawn.

[0101] Other changes and substitutions will be apparent to those skilledin the art of event detection from the description of the foregoingpreferred embodiments without departing from the spirit of the presentinvention, defined by the appended claims.

What is claimed is:
 1. A method for detecting matching records among aflow of records into a database, said method comprising the steps of:establishing a condition for use in selecting a set of matching records;applying a time stamp to each record in a flow of records as said eachrecord enters a database; incrementing said time stamp after applyingsaid time stamp to said each record so that said each record has adifferent time stamp; defining a sequence of time stamps from a firsttime stamp to a latest time stamp; defining a set of current recordsfrom records in said flow of records wherein each record in said set ofcurrent records has a time stamp falling between said first time stampand said latest time stamp; applying said condition to said database tofind a set of matching records wherein said set of matching recordsincludes at least one current record from said set of current recordsand no records having a time stamp greater than said latest time stamp;and outputting said matching records.
 2. The method as recited in claim1, further comprising the step, following said condition applying step,of redefining said sequence of time stamps wherein said latest timestamp becomes said first time stamp and a later time stamp becomes saidlatest time stamp.
 3. The method as recited in claim 1, wherein saidtime stamp applying step further comprises the steps of: applying saidtime stamp to said record; and then inserting said record into saiddatabase.
 4. The method as recited in claim 1, wherein said time stampapplying step further comprises the steps of: inserting said record intosaid database; applying said time stamp to said record while blockingsaid condition applying step until said time stamp is applied to saidrecord.
 5. The method as recited in claim 1, wherein said databaseincludes plural tables, said each record being inserted into one tableof said plural tables, and wherein said matching records include at mostone record from said one table and at most one record from another tableof said plural tables.
 6. A method for detecting matching records amonga flow of records into a database, said method comprising the steps of:establishing an event condition; establishing a latest variable, an oldvariable and a new variable; setting said new variable to a value ofzero; receiving a record from a flow of records; augmenting said recordwith a time stamp; replacing the value of said latest variable with saidtimestamp; replacing the value of said old variable with the value ofsaid new variable; replacing the value of said new variable with thevalue of said latest variable; inserting said augmented record into adatabase; and finding all matches among records in said database forsaid event condition that have at least one record with a timestampgreater than said old time stamp and no records with time stamps greaterthan said new time stamp.
 7. A system for detecting records that meetpre-selected conditions, said system comprising: means for creating aflow of records; a database for receiving each record in said flow ofrecords; time stamp manager means for issuing a time stamp to said eachrecord entering said database and for incrementing said time stamp;means for establishing a range of time stamps beginning with a firsttime stamp and ending with a latest time stamp; means for storing apreselected condition; condition manager means for applying apre-selected condition to each record in said flow of records having atime stamp in said range of time stamps in order to find a current matchbetween a record having a time stamp within said range and a recordwithin said flow of records; and means for outputting said currentmatch.
 8. The system as recited in claim 7, wherein said time stampmanager applies said time stamps to said each record before said eachrecord enters said database.
 9. The system as recited in claim 7,wherein said time stamp manager increments said time stamp after issuingsaid time stamp to said each record.
 10. The system as recited in claim7, wherein said database includes plural tables and wherein said systemfurther comprises means for collecting current matches from said pluraltables.